Saliency-Enhanced Content-Based Image Retrieval for Diagnosis Support in Dermatology Consultation: Reader Study.

BACKGROUND Previous research studies have demonstrated that medical content image retrieval can play an important role by assisting dermatologists in skin lesion diagnosis. However, current state-of-the-art approaches have not been adopted in routine consultation, partly due to the lack of interpretability limiting trust by clinical users. OBJECTIVE This study developed a new image retrieval architecture for polarized or dermoscopic imaging guided by interpretable saliency maps. This approach provides better feature extraction, leading to better quantitative retrieval performance as well as providing interpretability for an eventual real-world implementation. METHODS Content-based image retrieval (CBIR) algorithms rely on the comparison of image features embedded by convolutional neural network (CNN) against a labeled data set. Saliency maps are computer vision-interpretable methods that highlight the most relevant regions for the prediction made by a neural network. By introducing a fine-tuning stage that includes saliency maps to guide feature extraction, the accuracy of image retrieval is optimized. We refer to this approach as saliency-enhanced CBIR (SE-CBIR). A reader study was designed at the University Hospital Zurich Dermatology Clinic to evaluate SE-CBIR's retrieval accuracy as well as the impact of the participant's confidence on the diagnosis. RESULTS SE-CBIR improved the retrieval accuracy by 7% (77% vs 84%) when doing single-lesion retrieval against traditional CBIR. The reader study showed an overall increase in classification accuracy of 22% (62% vs 84%) when the participant is provided with SE-CBIR retrieved images. In addition, the overall confidence in the lesion's diagnosis increased by 24%. Finally, the use of SE-CBIR as a support tool helped the participants reduce the number of nonmelanoma lesions previously diagnosed as melanoma (overdiagnosis) by 53%. CONCLUSIONS SE-CBIR presents better retrieval accuracy compared to traditional CBIR CNN-based approaches. Furthermore, we have shown how these support tools can help dermatologists and residents improve diagnosis accuracy and confidence. Additionally, by introducing interpretable methods, we should expect increased acceptance and use of these tools in routine consultation

Saliency-Enhanced Content-Based Image Retrieval for Diagnosis Support in Dermatology Consultation: Reader Study

BACKGROUND Previous research studies have demonstrated that medical content image retrieval can play an important role by assisting dermatologists in skin lesion diagnosis. However, current state-of-the-art approaches have not been adopted in routine consultation, partly due to the lack of interpretability limiting trust by clinical users. OBJECTIVE This study developed a new image retrieval architecture for polarized or dermoscopic imaging guided by interpretable saliency maps. This approach provides better feature extraction, leading to better quantitative retrieval performance as well as providing interpretability for an eventual real-world implementation. METHODS Content-based image retrieval (CBIR) algorithms rely on the comparison of image features embedded by convolutional neural network (CNN) against a labeled data set. Saliency maps are computer vision-interpretable methods that highlight the most relevant regions for the prediction made by a neural network. By introducing a fine-tuning stage that includes saliency maps to guide feature extraction, the accuracy of image retrieval is optimized. We refer to this approach as saliency-enhanced CBIR (SE-CBIR). A reader study was designed at the University Hospital Zurich Dermatology Clinic to evaluate SE-CBIR's retrieval accuracy as well as the impact of the participant's confidence on the diagnosis. RESULTS SE-CBIR improved the retrieval accuracy by 7% (77% vs 84%) when doing single-lesion retrieval against traditional CBIR. The reader study showed an overall increase in classification accuracy of 22% (62% vs 84%) when the participant is provided with SE-CBIR retrieved images. In addition, the overall confidence in the lesion's diagnosis increased by 24%. Finally, the use of SE-CBIR as a support tool helped the participants reduce the number of nonmelanoma lesions previously diagnosed as melanoma (overdiagnosis) by 53%. CONCLUSIONS SE-CBIR presents better retrieval accuracy compared to traditional CBIR CNN-based approaches. Furthermore, we have shown how these support tools can help dermatologists and residents improve diagnosis accuracy and confidence. Additionally, by introducing interpretable methods, we should expect increased acceptance and use of these tools in routine consultation

Dreidimensionale Fluoreszenzbildanalyse von Megakaryozyten und Gefäßstrukturen in intaktem Knochen

The thesis provides insights in reconstruction and analysis pipelines for processing of three-dimensional cell and vessel images of megakaryopoiesis in intact murine bone. The images were captured in a Light Sheet Fluorescence Microscope. The work presented here is part of Collaborative Research Centre (CRC) 688 (project B07) of the University of Würzburg, performed at the Rudolf-Virchow Center. Despite ongoing research within the field of megakaryopoiesis, its spatio-temporal pattern of megakaryopoiesis is largely unknown. Deeper insight to this field is highly desirable to promote development of new therapeutic strategies for conditions related to thrombocytopathy as well as thrombocytopenia. The current concept of megakaryopoiesis is largely based on data from cryosectioning or in vitro studies indicating the existence of spatial niches within the bone marrow where specific stages of megakaryopoiesis take place. Since classic imaging of bone sections is typically limited to selective two-dimensional views and prone to cutting artefacts, imaging of intact murine bone is highly desired. However, this has its own challenges to meet, particularly in image reconstruction. Here, I worked on processing pipelines to account for irregular specimen staining or attenuation as well as the extreme heterogeneity of megakaryocyte morphology. Specific challenges for imaging and image reconstruction are tackled and solution strategies as well as remaining limitations are presented and discussed. Fortunately, modern image processing and segmentation strongly benefits from continuous advances in hardware as well as software-development. This thesis exemplifies how a combined effort in biomedicine, computer vision, data processing and image technology leads to deeper understanding of megakaryopoiesis. Tailored imaging pipelines significantly helped elucidating that the large megakaryocytes are broadly distributed throughout the bone marrow facing a surprisingly dense vessel network. No evidence was found for spatial niches in the bone marrow, eventually resulting in a revised model of megakaryopoiesis.Im Rahmen dieses Dissertationsvorhabens wurden Segmentierungs- und Auswertepipelines dreidimensionaler Bilder von Zellen und Gefäßen im intakten Mausknochen erarbeitet. Die Bilder entstanden durch Fluoreszenzaufnahmen eines Lichtblattmikroskops. Das Dissertationsvorhaben war Teil des Sonderforschungsbereichs 688 (Teilprojekts B07) der Universität Würzburg und es wurde am Rudolf-Virchow-Zentrum durchgeführt. Trotz einer Vielzahl aktueller Forschungsprojekte auf dem Gebiet der Megakaryopoese sind Erkenntnisse über deren räumlich-zeitliche Zusammenhänge größtenteils unbekannt. Neuere wissenschaftliche Erkenntnisse auf diesem Gebiet wären insbesondere hilfreich für die Weiterentwicklung von Behandlungsstrategien für Patienten, die an Thrombozytopenien oder Thrombozytopathien leiden. Das aktuell vorherrschende Modell zur Erklärung der Megakaryopoese geht von der Existenz räumlicher Nischen im Knochenmark aus, in denen sich die einzelnen Schritte der Megakaryopoese vollziehen. Dieses Modell basiert hauptsächlich auf Auswertungen von Gefrierschnitten sowie in-vitro Experimenten. Da die klassische Bildgebung von Knochenschnitten nur auf eine bestimmte Anzahl zweidimensionaler Schnitte begrenzt ist und deren Qualität unter Schnittartefakten leidet, ist die Bildgebung des intakten Knochens von besonderem Interesse. Dennoch führt dies zu neuen Herausforderungen im Bereich der Bilddatenauswertung. Im vorliegenden Dissertationsvorhaben beschäftige ich mich in diesem Bereich mit der Erarbeitung von Auswerteprotokollen, welche beispielsweise den Einfluss unregelmäßiger Färbungen oder Signalabschwächungen sowie die extreme Heterogenität der Megakaryozytenmorphologie berücksichtigen. Spezifische Herausforderungen für die Bildgebung und Bildrekonstruktion werden in Angriff genommen und Lösungsstrategien sowie verbleibende Einschränkungen werden vorgestellt und diskutiert. Erfreulicherweise profitieren insbesondere die moderne Bildbearbeitung sowie die Objekterkennung in großem Ausmaß von fortlaufenden Entwicklungen aus dem Hard- sowie Softwarebereich. Dieses Dissertationsvorhaben zeigt auf exemplarische Art und Weise auf, wie gemeinsame Forschungsanstrengungen im Bereich der Biomedizin, des Maschinellen Sehens, der Datenverarbeitung sowie der Bildtechnologie zu einem tieferen Verständnis der Megakaryopoese führen. Die maßgeschneiderten Pipelines zur Bilddatenauswertung stützten letztlich die These, dass größere Megakaryozyten im Knochenmark breit verteilt sind und von einem überraschend dichten Gefäßnetz umgeben sind. Beweise für die Existenz räumlicher Nischen im Knochenmark konnten nicht gefunden warden. Dieses führte schließlich zur Vorstellung eines überarbeiteten Modells der Megakaryopoese

Thrombopoiesis is spatially regulated by the bone marrow vasculature

In mammals, megakaryocytes (MKs) in the bone marrow (BM) produce blood platelets, required for hemostasis and thrombosis. MKs originate from hematopoietic stem cells and are thought to migrate from an endosteal niche towards the vascular sinusoids during their maturation. Through imaging of MKs in the intact BM, here we show that MKs can be found within the entire BM, without a bias towards bone-distant regions. By combining in vivo two-photon microscopy and in situ light-sheet fluorescence microscopy with computational simulations, we reveal surprisingly slow MK migration, limited intervascular space, and a vessel-biased MK pool. These data challenge the current thrombopoiesis model of MK migration and support a modified model, where MKs at sinusoids are replenished by sinusoidal precursors rather than cells from a distant periostic niche. As MKs do not need to migrate to reach the vessel, therapies to increase MK numbers might be sufficient to raise platelet counts

SARS-CoV-2 in Environmental Samples of Quarantined Households

The role of environmental transmission of SARS-CoV-2 remains unclear. Thus, the aim of this study was to investigate whether viral contamination of air, wastewater, and surfaces in quarantined households result in a higher risk for exposed persons. For this study, a source population of 21 households under quarantine conditions with at least one person who tested positive for SARS-CoV-2 RNA were randomly selected from a community in North Rhine-Westphalia in March 2020. All individuals living in these households participated in this study and provided throat swabs for analysis. Air and wastewater samples and surface swabs were obtained from each household and analysed using qRT-PCR. Positive swabs were further cultured to analyse for viral infectivity. Out of all the 43 tested adults, 26 (60.47%) tested positive using qRT-PCR. All 15 air samples were qRT-PCR-negative. In total, 10 out of 66 wastewater samples were positive for SARS-CoV-2 (15.15%) and 4 out of 119 surface samples (3.36%). No statistically significant correlation between qRT-PCR-positive environmental samples and the extent of the spread of infection between household members was observed. No infectious virus could be propagated under cell culture conditions. Taken together, our study demonstrates a low likelihood of transmission via surfaces. However, to definitively assess the importance of hygienic behavioural measures in the reduction of SARS-CoV-2 transmission, larger studies should be designed to determine the proportionate contribution of smear vs. droplet transmission